Formal Classification of Visual Languages

In terms of technical usability, the field of visual language specification has achieved a lot of progress since research into “syntactical pattern recognition” started. Several specification methods have been proposed and have proven useful in practical applications. This progress, however, came at a certain price. VL specification is now faced with a Babel of different formalisms that are very hard to compare. Most of them are grammar-like (such as Web and Array Grammars [15], Shape Grammars [7], Positional Grammars [4], Relational Grammars [6], Unification Grammars [18, 17], Attributed Multiset Grammars [8], Constraint Multiset Grammars [12], and several types of graph grammars [5]), but logic-based methods [10, 14, 9] and algebraical formalisms [16] are also used. Even when the discussion is restricted to grammar-like formalisms, there is such a diversity in the underlying assumptions that a comparison is made difficult. A systematic, comprehensive hierarchy of VLs based on formal properties – an analogue to the Chomsky hierarchy for textual languages — is still missing. Such a taxonomy is of great importance, because it does not only provide common ground for the comparison of results obtained from different approaches, but it can also yield new insights into the general structure of VLs. This paper presents the core of such a hierarchy based on Constraint Multiset Grammars, and some dimensions for extensions are discussed. Several classes of VLs are defined and their formal properties are analyzed. It is shown how the classification can serve to relate different specification methods to each other.